Skip to content

Refactored flaky tests in vertex-redis-client-3.9 to retry connection rather than fail with NPE #9390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
AlexeyKuznetsov-DD merged 3 commits into from
Aug 20, 2025
Merged

Refactored flaky tests in vertex-redis-client-3.9 to retry connection rather than fail with NPE #9390

AlexeyKuznetsov-DD merged 3 commits into from
Aug 20, 2025

Conversation

@AlexeyKuznetsov-DD
Copy link
Contributor

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD commented Aug 19, 2025 •
edited
Loading

What Does This Do

Reworked connection logic for vertex-redis-client-3.9 test to allow connection retries.

Motivation

Tests in vertex-redis-client-3.9 periodically failing with NPE java.lang.NullPointerException: Cannot invoke method dbsize() on null object, to prevent this I've implemented logic to retry connection to Redis.

Additional Notes

Test is passing on CI due to Gradle retries we introduced on build level, but it is visible as flaky test on CI dashboard.

@AlexeyKuznetsov-DD
Updated Redis docker image to 6.2.19. Reworked connection logic for v…
4ea2e08 
…ertex-redis test to allow connection retries.
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD marked this pull request as ready for review August 19, 2025 23:11
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD requested a review from a team as a code owner August 19, 2025 23:11
@github-actions
Copy link
Contributor

github-actions bot commented Aug 19, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD self-assigned this Aug 19, 2025
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Aug 19, 2025
@pr-commenter
Copy link

pr-commenter bot commented Aug 19, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/vertex-redis-flaky-npe
git_commit_date 1755712814 1755713832
git_commit_sha 269bad9 e0700b8
release_version 1.53.0-SNAPSHOT~269bad9567 1.53.0-SNAPSHOT~e0700b8c5f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755715545 1755715545
ci_job_id 1090490560 1090490560
ci_pipeline_id 74290731 74290731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-5pa5mesm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-5pa5mesm 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057191
Total [baseline] (10.815 s) : 0, 10814766
Agent [candidate] (1.053 s) : 0, 1053415
Total [candidate] (10.854 s) : 0, 10854142
section appsec
Agent [baseline] (1.238 s) : 0, 1237994
Total [baseline] (10.871 s) : 0, 10871431
Agent [candidate] (1.223 s) : 0, 1222640
Total [candidate] (10.821 s) : 0, 10821319
section iast
Agent [baseline] (1.181 s) : 0, 1181074
Total [baseline] (10.962 s) : 0, 10962129
Agent [candidate] (1.178 s) : 0, 1177637
Total [candidate] (10.958 s) : 0, 10957558
section profiling
Agent [baseline] (1.199 s) : 0, 1198768
Total [baseline] (10.957 s) : 0, 10957452
Agent [candidate] (1.205 s) : 0, 1204693
Total [candidate] (10.909 s) : 0, 10908751
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.238 s 180.803 ms (17.1%)
Agent iast 1.181 s 123.883 ms (11.7%)
Agent profiling 1.199 s 141.576 ms (13.4%)
Total tracing 10.815 s -
Total appsec 10.871 s 56.665 ms (0.5%)
Total iast 10.962 s 147.363 ms (1.4%)
Total profiling 10.957 s 142.686 ms (1.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent appsec 1.223 s 169.225 ms (16.1%)
Agent iast 1.178 s 124.222 ms (11.8%)
Agent profiling 1.205 s 151.278 ms (14.4%)
Total tracing 10.854 s -
Total appsec 10.821 s -32.823 ms (-0.3%)
Total iast 10.958 s 103.416 ms (1.0%)
Total profiling 10.909 s 54.609 ms (0.5%) 
gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.476 ms) : 0, 1476
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (738.701 ms) : 0, 738701
BytebuddyAgent [candidate] (737.643 ms) : 0, 737643
GlobalTracer [baseline] (244.757 ms) : 0, 244757
GlobalTracer [candidate] (244.264 ms) : 0, 244264
AppSec [baseline] (30.386 ms) : 0, 30386
AppSec [candidate] (30.394 ms) : 0, 30394
Debugger [baseline] (6.073 ms) : 0, 6073
Debugger [candidate] (6.126 ms) : 0, 6126
Remote Config [baseline] (680.298 µs) : 0, 680
Remote Config [candidate] (679.214 µs) : 0, 679
Telemetry [baseline] (13.881 ms) : 0, 13881
Telemetry [candidate] (11.708 ms) : 0, 11708
section appsec
crashtracking [baseline] (1.479 ms) : 0, 1479
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (766.681 ms) : 0, 766681
BytebuddyAgent [candidate] (755.057 ms) : 0, 755057
GlobalTracer [baseline] (237.824 ms) : 0, 237824
GlobalTracer [candidate] (234.757 ms) : 0, 234757
IAST [baseline] (23.748 ms) : 0, 23748
IAST [candidate] (23.587 ms) : 0, 23587
AppSec [baseline] (171.169 ms) : 0, 171169
AppSec [candidate] (168.633 ms) : 0, 168633
Debugger [baseline] (5.774 ms) : 0, 5774
Debugger [candidate] (8.125 ms) : 0, 8125
Remote Config [baseline] (645.87 µs) : 0, 646
Remote Config [candidate] (643.628 µs) : 0, 644
Telemetry [baseline] (9.304 ms) : 0, 9304
Telemetry [candidate] (9.248 ms) : 0, 9248
section iast
crashtracking [baseline] (1.461 ms) : 0, 1461
crashtracking [candidate] (1.45 ms) : 0, 1450
BytebuddyAgent [baseline] (852.187 ms) : 0, 852187
BytebuddyAgent [candidate] (849.319 ms) : 0, 849319
GlobalTracer [baseline] (234.39 ms) : 0, 234390
GlobalTracer [candidate] (233.194 ms) : 0, 233194
IAST [baseline] (30.279 ms) : 0, 30279
IAST [candidate] (31.004 ms) : 0, 31004
AppSec [baseline] (26.13 ms) : 0, 26130
AppSec [candidate] (26.066 ms) : 0, 26066
Debugger [baseline] (6.644 ms) : 0, 6644
Debugger [candidate] (6.647 ms) : 0, 6647
Remote Config [baseline] (606.041 µs) : 0, 606
Remote Config [candidate] (605.971 µs) : 0, 606
Telemetry [baseline] (8.245 ms) : 0, 8245
Telemetry [candidate] (8.307 ms) : 0, 8307
section profiling
crashtracking [baseline] (1.432 ms) : 0, 1432
crashtracking [candidate] (1.437 ms) : 0, 1437
BytebuddyAgent [baseline] (763.54 ms) : 0, 763540
BytebuddyAgent [candidate] (767.728 ms) : 0, 767728
GlobalTracer [baseline] (222.618 ms) : 0, 222618
GlobalTracer [candidate] (222.573 ms) : 0, 222573
AppSec [baseline] (30.186 ms) : 0, 30186
AppSec [candidate] (30.337 ms) : 0, 30337
Debugger [baseline] (6.333 ms) : 0, 6333
Debugger [candidate] (6.374 ms) : 0, 6374
Remote Config [baseline] (707.094 µs) : 0, 707
Remote Config [candidate] (759.451 µs) : 0, 759
Telemetry [baseline] (16.334 ms) : 0, 16334
Telemetry [candidate] (16.451 ms) : 0, 16451
ProfilingAgent [baseline] (107.845 ms) : 0, 107845
ProfilingAgent [candidate] (109.1 ms) : 0, 109100
Profiling [baseline] (108.468 ms) : 0, 108468
Profiling [candidate] (109.763 ms) : 0, 109763
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.051 s) : 0, 1051146
Total [baseline] (8.65 s) : 0, 8649990
Agent [candidate] (1.047 s) : 0, 1047468
Total [candidate] (8.625 s) : 0, 8625492
section iast
Agent [baseline] (1.181 s) : 0, 1181227
Total [baseline] (9.346 s) : 0, 9346405
Agent [candidate] (1.181 s) : 0, 1180700
Total [candidate] (9.338 s) : 0, 9338136
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent iast 1.181 s 130.08 ms (12.4%)
Total tracing 8.65 s -
Total iast 9.346 s 696.415 ms (8.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.181 s 133.231 ms (12.7%)
Total tracing 8.625 s -
Total iast 9.338 s 712.644 ms (8.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (734.758 ms) : 0, 734758
BytebuddyAgent [candidate] (731.26 ms) : 0, 731260
GlobalTracer [baseline] (243.23 ms) : 0, 243230
GlobalTracer [candidate] (242.898 ms) : 0, 242898
AppSec [baseline] (30.16 ms) : 0, 30160
AppSec [candidate] (30.291 ms) : 0, 30291
Debugger [baseline] (6.081 ms) : 0, 6081
Debugger [candidate] (6.094 ms) : 0, 6094
Remote Config [baseline] (684.285 µs) : 0, 684
Remote Config [candidate] (690.547 µs) : 0, 691
Telemetry [baseline] (13.706 ms) : 0, 13706
Telemetry [candidate] (13.78 ms) : 0, 13780
section iast
crashtracking [baseline] (1.459 ms) : 0, 1459
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (851.75 ms) : 0, 851750
BytebuddyAgent [candidate] (852.007 ms) : 0, 852007
GlobalTracer [baseline] (233.586 ms) : 0, 233586
GlobalTracer [candidate] (233.508 ms) : 0, 233508
AppSec [baseline] (25.301 ms) : 0, 25301
AppSec [candidate] (27.786 ms) : 0, 27786
Debugger [baseline] (8.402 ms) : 0, 8402
Debugger [candidate] (5.735 ms) : 0, 5735
Remote Config [baseline] (612.691 µs) : 0, 613
Remote Config [candidate] (594.704 µs) : 0, 595
Telemetry [baseline] (8.306 ms) : 0, 8306
Telemetry [candidate] (8.33 ms) : 0, 8330
IAST [baseline] (30.699 ms) : 0, 30699
IAST [candidate] (30.364 ms) : 0, 30364
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/vertex-redis-flaky-npe
git_commit_date 1755712814 1755713832
git_commit_sha 269bad9 e0700b8
release_version 1.53.0-SNAPSHOT~269bad9567 1.53.0-SNAPSHOT~e0700b8c5f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755715215 1755715215
ci_job_id 1090490561 1090490561
ci_pipeline_id 74290731 74290731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-ujj9wn1t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-ujj9wn1t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 10 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:no_agent:high_load worse
[+0.904ms; +1.539ms] or [+2.513%; +4.282%]		 unstable
[-13.183op/s; +4.633op/s] or [-10.139%; +3.563%]		 37.174ms 125.750op/s 35.953ms 130.025op/s
scenario:load:petclinic:profiling:high_load worse
[+1.748ms; +2.672ms] or [+3.728%; +5.698%]		 unstable
[-10.630op/s; +4.244op/s] or [-10.788%; +4.307%]		 49.099ms 95.350op/s 46.889ms 98.543op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.448 ms) : 4398, 4498
.   : milestone, 4448,
iast (9.45 ms) : 9297, 9602
.   : milestone, 9450,
iast_FULL (14.097 ms) : 13818, 14376
.   : milestone, 14097,
iast_GLOBAL (10.204 ms) : 10026, 10381
.   : milestone, 10204,
profiling (8.76 ms) : 8616, 8904
.   : milestone, 8760,
tracing (7.955 ms) : 7841, 8068
.   : milestone, 7955,
section candidate
no_agent (4.367 ms) : 4317, 4416
.   : milestone, 4367,
iast (9.588 ms) : 9428, 9748
.   : milestone, 9588,
iast_FULL (14.412 ms) : 14129, 14695
.   : milestone, 14412,
iast_GLOBAL (10.449 ms) : 10267, 10632
.   : milestone, 10449,
profiling (8.966 ms) : 8802, 9130
.   : milestone, 8966,
tracing (7.936 ms) : 7812, 8059
.   : milestone, 7936,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.448 ms [4.398 ms, 4.498 ms] -
iast 9.45 ms [9.297 ms, 9.602 ms] 5.001 ms (112.4%)
iast_FULL 14.097 ms [13.818 ms, 14.376 ms] 9.649 ms (216.9%)
iast_GLOBAL 10.204 ms [10.026 ms, 10.381 ms] 5.756 ms (129.4%)
profiling 8.76 ms [8.616 ms, 8.904 ms] 4.312 ms (96.9%)
tracing 7.955 ms [7.841 ms, 8.068 ms] 3.507 ms (78.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.367 ms [4.317 ms, 4.416 ms] -
iast 9.588 ms [9.428 ms, 9.748 ms] 5.222 ms (119.6%)
iast_FULL 14.412 ms [14.129 ms, 14.695 ms] 10.045 ms (230.0%)
iast_GLOBAL 10.449 ms [10.267 ms, 10.632 ms] 6.083 ms (139.3%)
profiling 8.966 ms [8.802 ms, 9.13 ms] 4.599 ms (105.3%)
tracing 7.936 ms [7.812 ms, 8.059 ms] 3.569 ms (81.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567
    dateFormat X
    axisFormat %s
section baseline
no_agent (35.953 ms) : 35669, 36236
.   : milestone, 35953,
appsec (48.228 ms) : 47805, 48651
.   : milestone, 48228,
code_origins (44.637 ms) : 44252, 45022
.   : milestone, 44637,
iast (45.608 ms) : 45212, 46005
.   : milestone, 45608,
profiling (46.889 ms) : 46467, 47311
.   : milestone, 46889,
tracing (42.585 ms) : 42222, 42948
.   : milestone, 42585,
section candidate
no_agent (37.174 ms) : 36868, 37481
.   : milestone, 37174,
appsec (46.907 ms) : 46494, 47319
.   : milestone, 46907,
code_origins (44.378 ms) : 43991, 44766
.   : milestone, 44378,
iast (44.805 ms) : 44416, 45194
.   : milestone, 44805,
profiling (49.099 ms) : 48663, 49535
.   : milestone, 49099,
tracing (43.804 ms) : 43431, 44177
.   : milestone, 43804,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 35.953 ms [35.669 ms, 36.236 ms] -
appsec 48.228 ms [47.805 ms, 48.651 ms] 12.276 ms (34.1%)
code_origins 44.637 ms [44.252 ms, 45.022 ms] 8.684 ms (24.2%)
iast 45.608 ms [45.212 ms, 46.005 ms] 9.656 ms (26.9%)
profiling 46.889 ms [46.467 ms, 47.311 ms] 10.937 ms (30.4%)
tracing 42.585 ms [42.222 ms, 42.948 ms] 6.632 ms (18.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.174 ms [36.868 ms, 37.481 ms] -
appsec 46.907 ms [46.494 ms, 47.319 ms] 9.733 ms (26.2%)
code_origins 44.378 ms [43.991 ms, 44.766 ms] 7.204 ms (19.4%)
iast 44.805 ms [44.416 ms, 45.194 ms] 7.631 ms (20.5%)
profiling 49.099 ms [48.663 ms, 49.535 ms] 11.925 ms (32.1%)
tracing 43.804 ms [43.431 ms, 44.177 ms] 6.63 ms (17.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alexeyk/vertex-redis-flaky-npe
git_commit_date 1755712814 1755713832
git_commit_sha 269bad9 e0700b8
release_version 1.53.0-SNAPSHOT~269bad9567 1.53.0-SNAPSHOT~e0700b8c5f
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1755715835 1755715835
ci_job_id 1090490562 1090490562
ci_pipeline_id 74290731 74290731
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-4exk4z40 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-4exk4z40 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.249 s) : 15249000, 15249000
.   : milestone, 15249000,
appsec (14.975 s) : 14975000, 14975000
.   : milestone, 14975000,
iast (18.225 s) : 18225000, 18225000
.   : milestone, 18225000,
iast_GLOBAL (18.167 s) : 18167000, 18167000
.   : milestone, 18167000,
profiling (15.105 s) : 15105000, 15105000
.   : milestone, 15105000,
tracing (15.087 s) : 15087000, 15087000
.   : milestone, 15087000,
section candidate
no_agent (15.14 s) : 15140000, 15140000
.   : milestone, 15140000,
appsec (14.812 s) : 14812000, 14812000
.   : milestone, 14812000,
iast (18.572 s) : 18572000, 18572000
.   : milestone, 18572000,
iast_GLOBAL (18.334 s) : 18334000, 18334000
.   : milestone, 18334000,
profiling (15.286 s) : 15286000, 15286000
.   : milestone, 15286000,
tracing (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.249 s [15.249 s, 15.249 s] -
appsec 14.975 s [14.975 s, 14.975 s] -274.0 ms (-1.8%)
iast 18.225 s [18.225 s, 18.225 s] 2.976 s (19.5%)
iast_GLOBAL 18.167 s [18.167 s, 18.167 s] 2.918 s (19.1%)
profiling 15.105 s [15.105 s, 15.105 s] -144.0 ms (-0.9%)
tracing 15.087 s [15.087 s, 15.087 s] -162.0 ms (-1.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.14 s [15.14 s, 15.14 s] -
appsec 14.812 s [14.812 s, 14.812 s] -328.0 ms (-2.2%)
iast 18.572 s [18.572 s, 18.572 s] 3.432 s (22.7%)
iast_GLOBAL 18.334 s [18.334 s, 18.334 s] 3.194 s (21.1%)
profiling 15.286 s [15.286 s, 15.286 s] 146.0 ms (1.0%)
tracing 15.156 s [15.156 s, 15.156 s] 16.0 ms (0.1%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~e0700b8c5f, baseline=1.53.0-SNAPSHOT~269bad9567
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (3.696 ms) : 3477, 3915
.   : milestone, 3696,
iast (2.217 ms) : 2153, 2281
.   : milestone, 2217,
iast_GLOBAL (2.257 ms) : 2194, 2321
.   : milestone, 2257,
profiling (2.073 ms) : 2020, 2125
.   : milestone, 2073,
tracing (2.032 ms) : 1982, 2081
.   : milestone, 2032,
section candidate
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.698 ms) : 3478, 3917
.   : milestone, 3698,
iast (2.221 ms) : 2157, 2284
.   : milestone, 2221,
iast_GLOBAL (2.254 ms) : 2190, 2317
.   : milestone, 2254,
profiling (2.067 ms) : 2015, 2118
.   : milestone, 2067,
tracing (2.047 ms) : 1997, 2097
.   : milestone, 2047,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 3.696 ms [3.477 ms, 3.915 ms] 2.216 ms (149.7%)
iast 2.217 ms [2.153 ms, 2.281 ms] 736.964 µs (49.8%)
iast_GLOBAL 2.257 ms [2.194 ms, 2.321 ms] 776.818 µs (52.5%)
profiling 2.073 ms [2.02 ms, 2.125 ms] 592.349 µs (40.0%)
tracing 2.032 ms [1.982 ms, 2.081 ms] 551.475 µs (37.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 3.698 ms [3.478 ms, 3.917 ms] 2.221 ms (150.4%)
iast 2.221 ms [2.157 ms, 2.284 ms] 743.955 µs (50.4%)
iast_GLOBAL 2.254 ms [2.19 ms, 2.317 ms] 776.868 µs (52.6%)
profiling 2.067 ms [2.015 ms, 2.118 ms] 589.663 µs (39.9%)
tracing 2.047 ms [1.997 ms, 2.097 ms] 570.218 µs (38.6%)

PerfectSlayer
PerfectSlayer approved these changes Aug 20, 2025
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the stability improvement

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD merged commit c2e8852 into master Aug 20, 2025
501 checks passed
@AlexeyKuznetsov-DD AlexeyKuznetsov-DD deleted the alexeyk/vertex-redis-flaky-npe branch August 20, 2025 19:30
@github-actions github-actions bot added this to the 1.53.0 milestone Aug 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@sarahchen6 sarahchen6 sarahchen6 approved these changes

@amarziali amarziali Awaiting requested review from amarziali

Assignees

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.53.0

Development

Successfully merging this pull request may close these issues.

4 participants

@AlexeyKuznetsov-DD @PerfectSlayer @sarahchen6